ON THE PALAIS PRINCIPLE FOR NON-SMOOTH FUNCTIONALS 

MARCO SQUASSINA 

Abstract. If G is a compact Lie group acting linearly on a Banach space X and / is a 
G- invariant function on X. we provide some new versions of the so-called Palais' principle 
of symmetric criticality for / : X — > M. in the framework of non-smooth critical point 
— ^ theory. We apply the results to a class of quasi-linear PDEs associated with invariant 

fvq . functionals which are merely lower semi-continuous and thus could not be treated by 

previous non-smooth versions of the principle in the literature. 
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1. Introduction 



Let G be a compact Lie group which acts hnearly on a Banach space X, let / : X — )■ M 
cn \ be a G-invariant functional and consider the fixed point set of X under G, 

Fix(G') = Pi {n G X : (^ - Id)n = 0}, 
O ! gdG 

which is a closed subspace of X. In order to detect critical points of /, a possible approach 
is based upon what is currently known as Palais' principle of symmetric criticality, that 
rN \ is looking for critical points of /|Fix(G) in order to locate critical points of /. The origin 

c^ I of the principle is rather unclear and the first implicit use seems to trace back to 1950 

inside H. Weyl derivation of the Einstein field equations [26]. Next S. Coleman [5] 
makes an explicit reference to it around 1975. For G-invariant functionals of class C^ the 
principle was rigorously formulated by R.S. Palais in 1979 in a celebrated paper [17] 
(see also [11, 18, 19]), reading as 

\df\m^{G)[u) = 
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2 MARCO SQUASSINA 

In other words, in order for an invariant point u to be critical for /, it is sufficient that 
it be critical for /|Fix(G); namely if the directional derivative f'{u)v vanishes along any 
direction v tangent to Fix(G), then it must vanish along the directions transverse to 
Fix(G) as well. Although the implication is valid in a rather broad context, there are 
of course some pathological counterexamples showing that it can fail to hold (see [17, 
Section 3]). 

The validity of Palais' principle of symmetric criticality has a powerful impact on ap- 
plications to nonlinear problems of mathematical physics which are set on unbounded 
domains (for instance M^, half-spaces or strips), are invariant under some linear transfor- 
mations, such as rotations, thus exhibiting symmetry (for instance, spherical or cylindri- 
cal) and are associated with a suitable energy functional /. In fact, in this framework, it is 
often the case that the solutions of a PDE live in a Banach space X which is continuously 
embedded into a space Banach W but, unfortunately, the injection X ^^ W fails to be 
compact. Typically the unboundedness of the domain where the problem is set prevents 
these inclusions to be compact and the problem cannot be solved by standard methods of 
nonlinear analysis. On the contrary, in many interesting concrete situations, for suitable 
compact Lie groups G, the compactness of the embedding Fix(G) ^-> W is restored. We 
refer the reader to the classical works by P.L. Lions [14] and W.A. Strauss [24] (see 
for instance [27, Section 1.5] for a list of the results of [14, 24]). This fact offers a fruitful 
tool in order to find a critical point u of /|Fix(G) at the Mountain Pass level (hence a 
critical point of / by the principle and, in turn, a solution to the associated problem, in a 
suitable sense). In the language of non-smooth critical point theory the implication (1.1) 
corresponds to the property 

(1.2) fL%T'M ^ ''^^^'"'- 

[0 e 5/|Fix(G)(M) 

where df{u) is the subdifferential of / at u. In the current literature, some papers have 
already investigated the validity of this implication, for locally Lipschitz functions by 
W. Krawcewicz and W. Marzantowicz [12], for sums of C^ (resp. locally Lip- 
schitz) functions with lower semi-continuous convex functionals in the nice work by J. 
KOBAYASHI and M. Otani [10] (resp. by A. Kristaly, C. Varga and V. Varga [13]). 
On the other hand, a suitable definition of df{u) for any function / : X — >■ M has been 
recently given by I. Campa and M. Degiovanni in [3], consistently with the subdiffer- 
ential of convex functions and containing the subdifferential in the sense of F.H. Clarke 
[4] for locally Lipschitz functions. 

The main goal of this paper is to obtain two new Palais' symmetric criticality type 
principles for non-smooth functionals and furnish an application to elliptic PDEs associ- 
ated with merely lower semi-continuous functionals, thus out of the frame of the previous 
literature available on the subject. In Section 2 we will recall some definitions and tools of 
non-smooth analysis. In Section 3 we provide a first non-smooth version of the symmetric 
criticality principle. Theorem 3.2, for a class of functionals which includes, in particular, 
the cases covered by the previous literature (cf. Proposition 3.1). Theorem 3.2 asserts 
that, under suitable assumptions, property (1.2) holds true. In Section 4 we shall obtain 
a second abstract result in a very general framework (Theorem 4.1). The result proves. 
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under suitable assumptions, the implication 

(ueFix{G) 
(1.3) loedf\p,,^G){u) =^ Vt;eK: f'iu)v = 0. 

y3 Vu C X invariant, dense subspace 

In fact, in many concrete situations, the classical directional derivative exists on suitable 
dense subspaces of X, and the criticality of u over these spaces is often sufficient to 
guarantee that, actually, u is the solution of a corresponding PDE, in a generalized or 
distributional sense (cf. [20]). In Section 6, we shall discuss an application (Theorem 6.4) 
of Theorem 4.1 showing that, for an unbounded domain Q C M.^ invariant under a 
group Gn of rotations, under suitable assumptions there exists a nontrivial GTv-invariant 
solution to the quasi-linear problem 



t{u,\Du\)—^-Dv+ I js{u,\Du\)v+ I \u\^-\v= I 
In l-^^ Jn Jn Jn 



As a consequence (Corollary 6.5), in the case fl = M.^ , we will prove that this problem 
admits a radial solution, extending [27, Theorems 1.29] to the quasi-linear case. We 
would like to stress that the previous non-smooth versions of Palais' symmetric criticality 
principle in the literature [10, 13] cannot be used in order to prove the existence of 
solutions to the above elliptic problem, since its associated invariant functional is no 
more than lower semi-continuous (and non convex). For an overview on these classes of 
quasi-linear problems, we refer the interested reader to the monograph [23]. 
In contrast with the use of Palais's symmetric criticality principle discussed above, a 
direct approach is also possible, where, in the search of symmetric solutions, one avoids 
restricting the functional to Fix(G'). This approach has been recently developed by the 
author [21] for a class of lower semi-continuous functionals, extending previous results 
from [25] valid for C^ functionals. In Section 5 we state and prove a new abstract result. 
Theorem 5.1, yielding the existence of a Palais-Smale sequence (uh) C X for / at level c 
which is compact in a suitable Banach space W containing X, and it becomes more and 
more symmetric, as h increases. A final remark is now adequate. Let D and S are the 
closed unit ball and the sphere in M™ {m > 1) respectively and Pq C C(S, X), consider 
the (unrestricted) Mountain Pass energy level 

c := inf sup/(7(r)), P := {7 G C(B,X) : 7|s G Pq}. 

Then, setting 

pJ-(^) := {^er,: 7(§) C Fix(G)}, 
one can consider, for /|Fix(G) • Fix(G') — ?■ M, the restricted Mountain Pass energy level 

(1.4) c^^^(^) := inf sup/(7(r)), P^^^(^) := {7 G C(D,Fix(G)) : 7|s G P^^^^}. 

Then, automatically, it holds c < c^'^('^) in light of P^'^(^) c P. Hence, in general, the 
strategy based upon Palais' criticality principle ensures a simpler approach to concrete 
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problems but, as a drawback, the minimality property of the Mountain Pass energy level 
c might be lost, which is not the case in the above described direct approach. 

2. Some notions from non-smooth analysis 

In this section we consider abstract notions and results that will be used in the proof 
of the main result. For the definitions, we refer e.g. to [3, 6, 8]. Let X be a metric space, 
B{u, S) the open ball of center u and of radius S and let / : X — )■ M be a function. We set 

dom(/) = {ueX : f{u) < +00} and epi (/) = {(m, e X x R : f{u) < ^} . 

We recall the definition of the weak slope. 

Definition 2.1. For every u E X with f{u) G M, we denote by \df\{u) the supremum of 
a's in [0, 00) such that there exist 6 > and a continuous map 

H: i?((n,/(n)),(5)nepi(/)x[0,5]^X, 

satisfying 

dime, /i), t), < t, fimc, /i), t)) < /(o - at. 

for all {^, fi) G B{{u, f{u)), 6) fl epi(/) and t G [0, 6]. The extended real number \df\{u) is 
called the weak slope of f at u. 

Remark 2.2. Let X be a metric space, / : X — ?> R a continuous function, and m G X. 
Then \df\{u) is the supremum of the real numbers a in [0, 00) such that there exist d > 
and a continuous map "H : B{u, 6) x [0, 6] — )■ X, such that, for every v in B{u, 6), and for 
every t in [0, 6] it results 

d{n{v,t),v)<t, fmv,t))<f{v)-at. 

If furthermore X is an open subset of a normed space E and / is a function of class C^ 
on X, then \df\{u) = \\df{u)\\, for every n G X (see [8, Corollary 2.12]). 

Let us define the function Qf : epi (/) — )■ R by 

(2.1) GAu,0 = C 

In the following, epi (/) will be endowed with the metric 

so that the function Qf is Lipschitz continuous of constant 1. 
We have the following 

Proposition 2.3. For every u E X such that f{u) G R, we have 

\dQf\{u,f{u)) 

,' /'; ;\^ f==, tf dgf{u,f{u))<i, 

-00, if\dgf\{uj{u)) = l. 
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In order to apply the abstract theory to the study of lower semi-continuous functions, the 
following condition is crucial 

(2.2) v(n,OGepi(/): /(n) < ^ =^ |rf^^|(n,0 = l. 

We refer the reader to [6, 8] where this is discussed. 

Let now X and X* denote a real normed space and its topological dual, respectively. 
We recall [3, Definitions 4.3 and 5.5], namely the definition of the generalized directional 
derivative. 

Definition 2.4. Let u E X with f{u) G M. For every v E X and e > Q we define f°{u; v) 
to he the infimum of the r'sinM. such that there exist 5 > and a continuous function 

V : i?5(n,/(n)) nepi(/)x]0,(5] ^ i?,(tO, 

which satisfies 

f{^ + tV{{tfi),t))<fi + rt, 

whenever ({,/i) G Bs{u, f{u)) fl epi(/) and t e]0,6]. Finally, we set 

f°{u; v) := sup f°{u; v) = lim f°{u; v). 

We say that f°{u;v) is the directional derivative of f at u with respect to v. 

It is readily seen that the directional derivative f°{u]v) does not change if the norm of 
X is substituted by an equivalent one. 

According to [3, Definition 4.1, Corollary 4.7], we recall the notion of subdifferential. 

Definition 2.5. Let u E X with f{u) < +oo. We set 

dfiu) := [aEX* : {a,v) < f%u;v), \/v E X}. 

The set df{u) is convex and weakly* closed in X* . 

Again, the set df{u) does not change if the norm of X is substituted by an equivalent 
one. 

We conclude the section by recalling [3, Corollary 4.13 (ii)-(iii)]), establishing the connec- 
tion between the weak slope \df\{u) and the subdifferential df{u). 

Proposition 2.6. Let f : X -^ M. be a functional and u E X with f{u) E M. Assume 
that \df\{u) < +CX). Then df{u) ^ and 

min{||a|| -.aEdfiu)} < \df\{u). 

In particular, \df\{u) = implies G df{u). 

The previous notions allow us to give the following 

Definition 2.7. We say that u E dom(/) is a critical point of f if E df{u). Moreover, 
c eM. is a critical value of f if there exists a critical point u E dom(/) of f with f{u) = c. 

Definition 2.8. Let c G M. We say that f satisfies the Palais-Smale condition at level c 
({PS)c in short), if every sequence (m„,) in dom(/) such that G df{un) and f{un) -^ c 
admits a subsequence (un^.) converging in X. 
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3. A FIRST ABSTRACT RESULT 

Let G be a compact Lie group acting linearly on a real Banach space (X, || ■ ||). By 
suitably renorming X with an equivalent norm, we may assume without loss of generality 
that the action of G over X is isometric, that is 

Vu G X, \/g e G : \\gu\\ = \\u\\. 

For the proof, we refer the reader e.g. to [10, Proposition 3.15]. 

Let now /i denote the normalized Haar measure and define the map A : X ^ X, known 
as averaging map or barycenter map, providing the center of gravity of the orbit of a 
V EX,hj 

Vf G X : Av := gvdfi{g). 
Jg 

The map A enjoys some useful properties. Firstly, Av = v for all v G Fix(G). Moreover, it 

is a continuous linear projection from X onto Fix(G), by the left invariance of fi. Finally, 

if C C X is a closed, invariant (namely gG C G for all g E G), convex subset of X, then 

A{G) C G. See for instance [17, Section 5]. 

3.0.1. The statement. We consider the following assumption on /: 
We assume that for all m, f G Fix(G) there exist p > and C > with 

( f{c+t{z-Az))-m 

(3.1) <^ t >-C\\z-Az\\, 

[vCG5p(u)nFix(G), yzeBp{v), VtG]0,p] with f{C-tAz)< f{u) + p. 

As stated next, condition (3.1) includes, in particular, the classes of functions already 
considered in the previous literature [10, 13] on the subject. More precisely, we have the 
following 

Proposition 3.1. Let / : X — ;> M U {+00} be such that / = /o + /i, vohere /o : X — )■ M 
is a locally lipschitz function and /i : X — t- M U {+C)o} is a convex, proper, lower semi- 
continuous and G-invariant function. Then condition (3.1) is satisfied. 

Proof. Condition (3.1) can be checked independently for /o and /i. The proof for /q is 
trivial. Let us turn to the proof for /i. For every ( G Fix(G), z G X and t > 0, we 
consider the set 

G:={r^eX:fM<MC + t{z-Az))}. 

Of course G is convex and closed, since /i is convex and lower semi-continuous. Moreover, 
G is G-invariant too, since /i is G-invariant. Then, A{G) C G. Recalling that C G Fix(G), 
we have ( = A{C + t{z - Az)) G G, namely /i(C + t{z - Az)) - /i(C) > 0, which 
yields (3.1). D 

The main result of the section is the following 

Theorem 3.2. Let G he a compact Lie group acting linearly on X, let f : X ^ W be 
a G-invariant functional satisfying condition (3.1) and u G Fix(G) with f{u) G M 6e a 
critical point ci//|Fix(G)- Then u is a critical point of f . 



ON THE PALAIS PRINCIPLE FOR NON-SMOOTH FUNCTIONALS 7 

The theorem is obtained in a quite simple fashion via the abstract machinery developed 
in [3] and which does not actually involve any a-priori regularity assumption on the map 
/. Assumption (3.1) only enters in the proof of Proposition 3.5. 

3.0.2. Some preliminary results. Next we state some preparatory results for Theorem 3.2. 

Proposition 3.3. Let G be a compact Lie group acting linearly on X, let f : X ^ M. be 
a G -invariant functional and u G Fix(G) with f{u) G M. Then 

WveX,\/geG: r{u-v) = r{u-gv). 

Proof. As recalled, the action of G over X can be assumed to be isometric. Given v E X 
and (? G G, fix £ > 0. Let r G M such that there exist 5 > and a continuous function 

V : Bs{u, f{u)) n epi(/) x]0, 5] ^ B,{v), 

according to Definition 2.4. Since u G Fix(G) and / is G-invariant, if (^, /i) G Bs{u, f{u))r\ 
epi(/), then also {g^, ji) G Bs{u, f{u)) fl epi(/) as H^f^ - ^H = \\9^ - gu\\ = ||^ - u\\ < 6 
and f{g^) = /(O < /^- Then, we are allowed to consider the continuous function 

V : Bs{u, f{u)) n epi(/) x]0, 5] ^ B,{g-\,), 

defined by setting 

V((e,/i),t):=^"V(((7e,/i),t) 

for all (^, /i) G B^iu, f{u)) fl epi(/) and t g]0, S\. It follows that V is well defined as 

II V((e, /x), t) - g-hi = \\g-'V{ig^, /x), t) - g~'v\\ = || V((^e, /x), t) - t;|| < e. 
Furthermore, it follows 

/(e + tvm /i), t)) = /(e + tg-'vm, ^^),t)) 

= f{g^ + tV{{g^,fi),t))<fi + rt, 

for all (^,/i) G Bs{u, f{u)) nepi(/) and t g]0, S\. By definition, it follows f^{u; g~^v) < r. 
By the arbitrariness of r, f°{u] g~^v) < f°{u\v). In a similar fashion, the opposite 
inequality follows as well. In conclusion, f°{u;g~^v) = f°{u;v) for every e > 0. The 
assertion follows by letting e — )■ 0. D 

We now have the following 

Proposition 3.4. Let G be a compact Lie group acting linearly on X , let f : X ^ M. be 
a G -invariant functional and u G Fix(G') with f{u) G M. Assume that 

VwGFix(G): f°{u;v)>0. 

Then 

yveX: f°{u;v)>0. 

Proof. Let e > and consider the set 

G, = {veX:f°{u,v)<-s}. 

Assume, by contradiction, that Gs 7^ 0. By [3, Corollary 4.6] the map {v i-)- f°{u]v)} 
is convex and lower semi- continuous. Then G^ is convex and closed. Furthermore G^ is 
G-invariant in X by means of Proposition 3.3. In turn, A{G^) C G^ fl Fix(G). Since 
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A{Ce) 7^ 0, we deduce that C^ fl Fix(G') 7^ 0, which yields a contradiction. By the 
arbitrariness of e, 

{veX:nu,v)<0} = [jCe = ^, 

£>0 

which proves the assertion. D 

The next assertion is the crucial step in order to link the generalized directional deriva- 
tives of /|Fix(G) with the generalized directional derivatives of / at a point u G Fix(G). 

Proposition 3.5. Let G be a compact Lie group acting linearly on X, let f : X ^M. he 
a functional which satisfies (3.1) and u G Fix(G) with f{u) G M. Then 

Wv G Fix(G') : r (n; v) > (/|Fix(G))°(^; v). 

In particular, if 

WveFix{G): (/|Fix(G))°(«;^)>0, 
then 

VwGFix(G): f°{u;v)>0. 

Proof. Let v G Fix(G) and let e G (0,p), where p > is the number appearing in 
assumption (3.1) on /. Moreover, let r G M be such that there exist 6 > and a 
continuous function 

V : Bs{u, f{u)) nepi{f)x]0,5] ^ B,{v), 
according to Definition 2.4. Then, we choose a number 

(3.2) 0<^-<min{^, ^ ), 

I- 1 + 6+ \\v\\ ) 

and we define a map 

V : Bs.{u, f{u)) n epi(/) n (Fix(G) x M)x]0, 6'] -^ B,{v) n Fix(G'), 

by setting 

V((e,/i),t):=AV((e,/i),t), 
forall(^,/i) G Bs'iuJ{u))nepi{f)n{Fix{G)xR) andt g]0,5']. Notice that, as \\A\\ < 1, 

||V((e,/i),t) - v\\ = \\AV{{^,^^),t) - Av\\ < P||||V((e,p),t) - i;|| < e, 

for all (^, p) G Bs'iu, /(«)) nepi(/) n (Fix(G) x M) and t g]0, 6'], so that V is well defined. 
We shall use assumption (3.1), applied with 

C:=^ + tAz, ^:=V((e,/i),t), (e,/i)G55,(M,/(M))nepi(/)nFix(G)xM, tG]0,5']. 

Notice that C e Fix(G), /(C - tAz) = /(O < p < f{u) + 6' < f{u) + p and 

(3.3) 11^ - Az\\ = II V((e, /i), t) - AV((e, /i), t) II 

<\\V{{^,fi),t)-v\\ + \\Av-AV{{^,p),t)\\ 
<e{l + \\A\\)<2e. 

Moreover, ||2; — f || = ||V((^,yU.), t) — v\\ < e < p and, by the choice in (3.2), 

IK -^11 < IK-^^ll+^ll^lllkll <S' + S'\\z\\ <6'{l+e+\\v\\) <p. 
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Then, for (^,/i) e Bs'{uJ{u)) n epi(/) n (Fix(G) x M) and t e]0,5'], by (3.1) and (3.3), 

/(e + tv((e, /i), t)) = /(e + tAva^, /x), t)) = /(o 

< /(C + t{z - Az)) + Ct||z - Az\\ 

</(e + tv((e,/i),t)) + 2C£t 

< /i + (r + 2Ce)t. 

It follows r + 2C£: > (/|Fix(G))°('w; '^)- By the arbitrariness of r G M, it follows 

f:{u;v)>if\p;^^G))>;v)-2Ce. 
Finally, by the arbitrariness of e, letting 5 — > O"*", we conclude 

f°{u;v) > (/|Fix(G))°(M;^^), 

which yields the assertion. D 

3.0.3. Proof of Theorem 3.2. Let u G Fix(G') be a critical point of the function /|Fix(G), 
that is G 5(/|Fix(G) )("")• Then (/|Fix(G))°(w; f) > for every v G Fix(G), by Defini- 
tion 2.5. In light of Proposition 3.5, we have f°{u]v) > for all v G Fix(G). Finally, 
by Proposition 3.4, we get f°{u] v) > for all v G X, namely G df{u), so that m is a 
critical point of /. D 

Remark 3.6. In the proof of Proposition 3.5, given the deformation V, in order to build a 
new deformation V that satisfies the desired properties, the idea is to compose V with A to 
ensure the values of V are projected into Fix(G). This, at the end, requires the technical 
assumption (3.1). In some concrete situations, the starting deformation V, when restricted 
to Fix(G) X M, automatically brings the values into Fix(G), so that one can take directly 
V = V|Fix(G)xR- See, for instance, the argument in the proof of Lemma 6.10. 

4. A SECOND ABSTRACT RESULT 

Let G be a compact Lie group acting linearly on a real Banach space {X, \\ ■ ||). We 
assume that for every u G Fix(G) there exists a G-invariant dense vectorial subspace Vu 
of X such that the following conditions are satisfied 



(4.1) Vn G Fix(G), Wv G K 

(4.2) Vu G Fix(G), Vt; G K n Fix(G) 

(4.3) Vm G Fix(G), yv G K 



the directional derivative f'{u)v exists, 
{fWiG)nu;v) < f{u)v, 

nu-v)<nu)v. 



Furthermore, for every u G Fix(G), we assume that 

(4.4) ^ ~ IJ ^'ji ^j is convex, closed, G-invariant with f'{u)\cj continuous 

The main result of the section is the following 

Theorem 4.1. Let G he a compact Lie group acting linearly on a Banach space X and let 
f : X -^ M be a G-invariant functional. Assume that conditions (4.1)-(4.4) are satisfied 
and let u G Fix(G) with f{u) ^M. he a critical point of f\pix{G)- Then 

\/veVu: f'{u)v = 0. 
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Furthermore, df{u) = {0} provided that df{u) is nonempty. 

In the statement of Theorem 4.1 we are not exphcitly assuming any global regularity 
on the functional /. In the last section of the paper we shall apply it to a class of 
(nonconvex) lower semi-continuous functionals. This achievement could not be reached 
through previous non-smooth versions of Palais' symmetric criticality principle in the 
literature as they require / to be the sum of a locally Lipschitz functions with a lower 
semi-continuous convex function [13]. 

4.0.4. Proof of Theorem 4 ■ 1 ■ Let u G Fix(G') be a critical point of /|Fix(G); G c?(/|Fix(G))(^) 
Then {f\Fi^{G))°{u]v) > 0, for every v G Fix(G). In particular, {f\Fi^(G))°{u]v) > 0, for 
every w G V^ fl Fix(G). In turn, in light of (4.2), by exploiting the linearity of the map 
{v H- >■ f'{u)v}, we get 

Vi; G K n Fix(G) : f'{u)v = 0. 
Let now s > 0, j > 1 and consider the set 

Di = {ve C, : f\u)v < -e}. 

Assume, by contradiction, that D^ ^ 0. Of course, D^ is convex, closed and G-invariant 
in X (recall that assumption (4.4) holds). In turn, A{Di) C D^ n Fix(G). We deduce 
that Di n Fix(G) ^ 0, which yields a contradiction. By the arbitrariness of e and j, 

{veV^:f'{u)v<0} = [j[JDi=il}, 

jeJe>0 

yielding 

(4.5) Vt; G K : f{u)v = 0. 

This proves the first assertion. Concerning the second assertion, assume that df{u) is not 
empty and let a G X* with a G df{u). Then, in light of (4.3) and (4.5), we have 

\/v G K : (a, v) < f°{u; v) < f\u)v = 0. 

Then, 

Vi; G K : («, v) = 0. 
Taking into account that Vu is dense in X, we conclude a = 0. Hence df{u) C {0}. D 

5. A DIRECT APPROACH 

The main goal of this section is that of showing how a direct (or, say, unrestricted) ap- 
proach in the search of symmetric critical points can also be outlined. In this approach, 
the compactness of the Palais-Smale sequences in a suitable space (relevant for applica- 
tions to PDEs) is achieved by exploiting the symmetry properties of the functional instead 
of restricting the functional to a space of symmetric functions. After recalling an abstract 
symmetrization framework due to [25], we shall state and prove a general abstract result 
by using the main result from [21]. 
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5.1. Abstract symmetrization. We recall a definition from [25]. 

Let X and V be two Banach spaces and S G X. We consider two maps * : S -^ S, 
u ^ u* {symmetrization map) and h : S x Ti^ ^ S, {u,H) \-^ u^ {polarization map), 
where "H* is a path-connected topological space. We assume that the following conditions 
hold: 

(1) X is continuously embedded in V; 

(2) /i is a continuous mapping; 

(3) for each u E S and H E n^ it holds {u*)^ = {u^)* = u* and u"" = u"; 

(4) there exists a sequence {Hm) in "H* such that, for u E S, u^^'"^"^ converges to u* 
mV- 

(5) for every u,v E S and H ETi^. it holds \\u^ — v^\\v < \\u — v\\v- 
Furthermore * : S ^ S can be extended to the whole space X by setting u* := {Q{u))* for 
all u E X, where B : {X, \\ ■ ||y) — )■ {S, \\ ■ \\v) is a Lipschitz function such that Q\s = ld\s- 

5.2. Compactness of Palais-Smale sequences. The main result of the section is the 
following 

Theorem 5.1. Let X be a reflexive Banach space, S G X and * : S ^ S a symmetriza- 
tion satisfying the requirements of the abstract symmetrization framework. Let V and W 
be Banach spaces containing X such that 

i i' 

(a) the injections X ^-^V ^^W are continuous; 

Let / : X — 7- ]RU{+oo} be a lower semi- continuous function satisfying (2.2). Lei D andE> 
denote the closed unit ball and the sphere in M™ (m > 1) respectively and Fq C C{E>,X). 
Let us define 

F = {7GC(D,X): 7|seFo}. 

Assume that 

+00 > c = inf sup /(7(t)) > sup sup /(7o(t)) = a, 

7er ^gjj 7o6ro tgs 

and that the following conditions hold 

(b) for all H eH,, for all ueS: f{u") < f{u); 

(c) for all ^ eT there exists 7 G F and Hq E H^. such that 

7(©)C5, /o7</o7, 7lfeFo. 
Assume furthermore that 

(d) for each Palais-Smale sequence {uh) C X for f , {uh) is bounded in X ; 

(e) for each Palais-Smale sequence {uh) C X for f , (uj^) converges in W . 

Then there exist u E X and a Palais-Smale sequence {uh) C X for f at level c such that 

(1) Uh ^ u weakly in X, as h ^ 00; 

(2) Uh ^ u strongly in W , as h -^ 00; 

(3) if {4) -{5) of the abstract framework hold with W in place ofV, then u = u* in W . 

The theorem states the existence, under suitable assumptions, of a Palais-Smale sequence 
{uh) in X which is convergent in a Banach W larger than X to a symmetric element. In 
particular, of course, it does not claim the compactness of {uh) in the original space X . 
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Notice also that, in the cases where the space X is a Sobolev space defined over a smooth 
bounded domain Q C M^ and V, W are subcritical L^{Q) spaces then, by Relhch Theorem, 
the injection X "^ W is compact and the stated strong convergence in W automatically 
follows from the weak convergence in X. On the other hand, this is not the case for 
PDEs which present loss of compactness, such as those set on an unbounded domain. In 
this sense. Theorem 5.1 allows to avoid restricting the concrete functional to the space of 
radial functions and then use Palais' symmetric criticality principle studied in the previous 
sections. It also provides an alternative to concentration-compactness arguments [15, 16] 
under symmetry assumptions. It is quite easy to realize that, whenever the conclusion of 
the theorem holds, namely Uh ^ u weakly in X and strongly in W, then it is often the 
case that, in turn, Uh ^ u strongly in X (using f{uh) -^ c and \df\{uh) — ;■ as /i — > oo). 
See, for instance, the compactness argument inside the proof of Theorem 6.4. For an 
application of Theorem 5.1 in the case p = 2 and f & C^ (the semi-linear case), see [25, 
Theorem 4.5]. In a similar fashion, applications to lower semicontinuous functionals can 
be given, by arguing as in [21], up to suitable necessary modifications. As a meaningful 
concrete framework where Theorem 5.1 applies one can think, for instance, to the case 
where, ioi p < m < p* , 

H/i'P(M^) = S = X^V = L^r] LP*(M^) = V '^W = L"*(M^), 

with z, i' continuous injections. The polarization and symmetrization functions are defined 

as u^ = \u\^ and u* = \u\* and (l)-(5) are satisfied (cf [25]). 

5.3. Proof of Theorem 5.1. In [21], the author proved the following 

Theorem 5.2. Let X and V be two Banach spaces, S G X , * and H* satisfying the 
requirements of the abstract symmetrization framework. Let / : X — )■ M U {+00} a lower 
semi- continuous function satisfying {2.2). Let D and S denote the closed unit ball and the 
sphere in W^ (m > 1) respectively and Fq C C{E>,X). Let us define 

F = {7gC(D,X): TlseTo}. 

Assume that 

+00 > c = inf sup /(7(t)) > sup sup /(7o(t)) = a, 

Tsr ^gjj 7o6ro tgs 

and that 

\/H G H,, VueS: f{u") < f{u). 
Then, for every e G (0, (c — a)/3), every 5 > and 7 G F such that 

sup /(7(r)) < c + £, 7(D) C S, 7]^° G Fq for some Hq G H*, 

reD 

there exists u E X such that 

(5.1) c-2e< f{u) <c + 2e, \df\{u) < 3e/S, \\u- u*\\v <3{{1 + Ce)K + l)S, 

being K the norm of the embedding map i : X ^ V and Cq the Lipschitz constant of 6. 
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5.4. Proof of Theorem 5.1 concluded. By the definition of the minimax value c, by 
choosing e = Eh = h~'^, for each h > 1 there exists a curve 7/^ G F such that 

sup/(7h,(r)) < c + — . 

tGD IT' 

Then, by virtue of assumption (c), we can find a curve 7/i G F and Hq G "H* such that 

sup fmr)) < c + -^, %m C S, 7h|f e To. 

tGB "• 

Choose also 6 = 6h = h~^. In turn, recalling assumption (b), by Theorem 5.2 there 
exists a sequence (uh) C X such that f{uh) — >■ c and |(i/|(M/i) — )■ as /i — )■ oo, with the 
additional information that 

(5.2) llwft — wjjlly — )• 0, as /i — > oo. 

In particular, by Definition 2.8, (uh) is a Palais-Smale sequence for / at the level c. By 
assumption (d), (uh) is bounded in X. Since X is refiexive, there exists u E X such 
that, up to a subsequence, Uh ^ u weakly in X, as /i — )■ oo. Consider now the sequence 
(u^) of abstract symmetrizations of u^. By assumptions (a) and (e), it follows that (m^) 
converges strongly in W and V is continuously embedded into W . In particular, there 
exists V E X such that m^ — ?■ w in VT as /i — )> oo and, for all /i > 1, 

ll^h - ^^||w < ll^ft - ulWw + ll^h - ^'llw < C\\uh - ulWv + ||mJ; - v\\w, 

which yields u^ ^ v strongly in W , for h — )■ oo, on account of (5.2). Then, of course, we 
deduce that v = u, which yields Uh ^ u strongly in W. Finally, assume that conditions 
(4)-(5) of the abstract symmetrization framework holds also with W in place of V. Then 
it is readily seen that \\z* — w*\\w < C@\\z — w\\w for all z,w E X (cf [21, Remark 2.1]). 
In turn, for all /i > 1, we get 

\\u* — u\\w < 11""* — u*^\\w + \\u*h — UhWw + \\uh — u\\w 
< (1 + Ce)\\uh - u\\w + C\\ul - UhWv, 

which yields u = u* in W, by virtue of (5.2) and the limit u/j — )■ u in W. D 

6. Application to quasi-linear PDEs 

In this section, we shall consider some applications of the abstract Palais criticality 
principle developed in Section 4. 

6.0.1. Compatible invariant domains. In the following, the symbol Gn will denote a sub- 
group of ^{N), the orthogonal group over M^, with N >2. According to [27, Definition 
1.22], we consider the following 

Definition 6.1. For every y G M^ and r > 0, we set 

i,r,GN) ■= sup{n G N : 3gi,...,gn e Gn ■ i 7^ j ^ Br{giy) n Br{gjy) = 0}. 
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An open (possibly unbounded) subset Q ofM.^ is said to be invariant provided that gQ = Q, 
for all g G Gn- An invariant subset Q is said to be compatible with Gn provided that 

lim ^{y^r^Gjsi) = oo 

|!/|->cx> 

dist{y,n)<r 

for some positive number r. 

Throughout the rest of the paper we shall assume that Q C M^ with A^ > 2 is a (possibly 
unbounded) smooth domain which is invariant under the action of G^. Let 

X = <'P(fi), l<p<iV, \\u\\i,p={\\u\\l+\\Dury''. 

The action of G^ on Wq'^{Q) is defined in a standard fashion by setting 

\/u G Wo'^in), MgeGN-. {gu){x) := u{g~^x) for a.e. xeVl. 

We shall denote by Fix(G'Ar) the set of fixed points u oi X with respect to the action of 

Gn, namely gu = u for all g G Gn- 

By virtue of [10, Proposition 4.2] we have the following 

Lemma 6.2. Assume that Q is compatible with Gn- Then the embeddings 

(6.1) Fix(G^) -> L'"(fi), for all p < m < p* 

are compact. 

Remark 6.3. A particular, but important, case is contained in Lemma 6.2, namely 

e 

n = M^, Gn ■-= ^{Ni) X ^{N2) x ■ ■ ■ ^(Ne), iV = J^ iV,-, i> 1, N^ > 2. 

i=i 

In fact, M^ is compatible with this Gn (cf. [27, Corollary 1.25]). Then Lemma 6.2 yields 

Fixv^.i,p(KiV)(^(iVi) X ^(A/'a) X ■ ■ ■ ff{Ne)) ^ L™(fi), for all p < m < p\ 

with compact injection. In particular, Fixi4/i,p(]RiV)(^(A^)) M- L"^{Q) with compact injec- 
tion. 

6.0.2. Invariant quasi-linear functional. Let / : Wo'^{Q) — t- MU {+00} be the functional 



(6.2) f{u) = J{u) + J(m), J{u) := / j(w, l^^l), Ku) - 



uW I \u 



'n Jn P Jn Q 

where p < q < p*. Notice that / : Wq'^{Q) — )■ M is a C^ functional while, on the contrary, 
J : Wq'^{Q) — )■ RU{+oo} is, in general, merely lower semi-continuous, by Fatou's lemma, 
if J > 0. Moreover, / is invariant under Gn, being invariant under the action of 
that is 

Wu G W^'^in), -ig G ff{N) : f{gu) = f{u). 

We consider the following assumptions on j. We assume that, for every s in M, 
(6.3) [t I— 7- j(s,t)} is strictly convex and increasing. 
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Moreover, there exist a constant ao > and a positive increasing function a G C{^ such 
that, for every (s,t) G R x R+, it holds 

(6.4) 0L^t'<3{s,t)<a{\s\)tK 

The functions js{s,t) and jt{s,t) denote the derivatives of j{s,t) with respect to the 
variables s and t respectively. Regarding the function js{s,t), we assume that there exist 
two positive increasing functions f3,'-f ^ C(M.) and a positive constant R such that 

(6.5) \3s{s,t)\<(3{\s\)tP, for every s in M and alH e M+, 

(6.6) |jt(s,t)| <7(|s|)tP-\ for every s in M and alH e M+, 

(6.7) js{s,t)s > 0, for every s in M with |s| > R and all t G M^. 

It is readily seen that, without loss of generality, we can assume that 7 = a, up to a 
constant. Furthermore, we assume that there exist R' > and 6 > such that 

(6.8) qj{s, t) - j,(s, t)s - (1 + 5)ji(s, t)t > 0, for every s e^ with \s\>R' 
and all t G M'^. Finally, it holds 

(6.9) hm ^ = 0. 

In the above assumptions, / is merely lower semi-continuous. If a is bounded, then / 
becomes a continuous functional. Condition (6.7) is typical for these problems and plays 
a significant role in the verification of the Palais-Smale condition and in the regularity 
theory (cf. e.g. [23]). Condition (6.8) allows the Palais-Smale sequences to be bounded in 
iyQ'^(r2), while (6.9) guarantees that / admits a Mountain Pass geometry. 

6.0.3. Statement of the results. The main result of the section is the following 

Theorem 6.4. Assume that Vt is compatible with Gn- Then there exists a nontrivial 
solution u G Fix(G'Ar) to the quasi-linear problem 

Du 



(6.10) / jt{u,\Du\)—— ■ Dv + / j,{u,\Du\)v+ / \uY~\v = / |m|«~\w, 
in \Du\ Jq Jq Jq 

for every v G C^{Q). 

Furthermore, let Q = M^ with N >2. Then, we have the following 

Corollary 6.5. Problem (6.10) admits a nontrivial radial positive solution. 

Remark 6.6. Assume p = 2 and j{s,t) = y. Then, Corollary 6.5 reduces to the results 
due to Strauss [24] (see, for instance, [27, Theorem 1.29]). 

Remark 6.7. In place of \u\'^~^u, more general nonlinearities f{\x\,u) could be handled 
by Theorem 6.4. For instance, if il = R^ and p < q < p*, one could assume that, for all 
e > 0, there exists C^ > such that 

(6.11) \f{\x\,s)\<e\s\P~'^ + C,\s\''~\ for a.e. X G R^ and all s G R. 

Let Wl^'^iR^) denote Fix(G) when X = W^'P{R^). We claim that the map {u h^ 
f{\x\,u)} is completely continuous from lyg'^(R^) to its dual, as soon as the injection of 
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Wg'^{R^) into L%R^) is compact. In fact, let (uh) C W^'^{R^) be a bounded sequence. 
Then, up to a subsequence, (uh) converges weakly in W^'''^{M.'^) and strongly in L'^{M.^) to 
some function u G Wq'^IM.'^). Let e > and let Ce be the constant appearing in (6.11). 
Then, it is readily checked that, for all 77 > 0, there exist R = R{ri) > and C > 
(independent of e and 77) with 

< CsTj + eC. 



sup sup 


/ {f{x,Uh)- 


- f{x\,u))v 


h>l „gwi.P(RiV) 


Jr'^\b(o,r) 




lkl|i,P<l 







Therefore, taking into account that ||/(|x|,m/i) — f{\x\,u)\\J^q'(^Q,QJ^^^ — )■ as /i — t- 00, the 
desired claim follows by letting first h —> 00, then 77 — )■ 0"^ and, finally, e — )■ 0"^. 

6.0.4. Some preparatory facts. For all u G Wq'^{Q), define the space 

(6.12) Vu = {ve Wo'^ift) n L°°{Q) : u G L°°{{x G Q : v{x) ^ 0})}. 

The vector space Vu is dense in Wq'^^Q) (cf. [9]). The following proposition, easy to 
prove, shows that Kt is a good test space to differentiate non-smooth functionals under 
suitable growth conditions. In particular, as a consequence, the abstract condition (4.1) 
is fulfilled. 

Proposition 6.8. Assume conditions (6.4), (6.5) and (6.7). Then, for every u G Wq'^{^) 
with J{u) < +00 and every v G Vu we have 

js{u, \Du\)v G L\Q), jt{u, \Du\)^^ ■ Dv G L\Q), 

\Du\ 

with the agreement that jt{u, \Du\)-Mj^^ = when \Du\ = (in view of (6.7)^). Moreover, 
the function {t i— )■ J{u + tv)} is of class C^ and 

J\u){v) = / 3t{u, \Du\)-^ ■Dv+ j,{u, \Du\)v. 
Jn \Du\ J^ 

In particular, 

Du 



f'{u){v)= / j,{u,\Du\)-—-Dv+ / js{u,\Du\)v 



Q \Du\ " ' J^""' " "- • I ' ' ■- I ' ' 



IP-2,„, / u,|g-2 



ur uv — I \ur uv 



for every v & Vu- 

For all u G Fix(G'Ar) and any j > 1, let us now set 



Cj = {v & Wq'^^^Q) : |f I < j a.e. in fi and \u\ < j a.e. where v 7^ 0}. 
Then, we have the following 

Lemma 6.9. For all u G Fix(Gjv) and j > 1, the set Cj is convex, closed, G^-invariant 
with f'{u)\c continuous and 

00 
(6.13) Vu = [jC,. 

i=i 
In particular Vu is Gn -invariant. 
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Proof. Of course each Cj is convex. Moreover Cj is G-invariant. In fact, if v G Cj, then 
\{gv){x)\ = \v{g~^x)\ < j a.e. in Q, being the domain invariant under Gn- Moreover, if 
X G {gv 7^ 0}, then v{g~^x) ^ 0, so that \u{x)\ = \u{g~^x)\ < j, since u G Fix{Giy). 
Hence gv G Cj. Let us now prove that Cj is closed. If (vh) C Cj with Vh ^ v in WQ'^{fl), 
then up to a subsequence |f/i(x)| < j and Vh{x) — )■ v{x), as /i — )■ oo, yielding |f | < j a.e. in 
Q. Moreover, ii A = {v ^ 0}, then by the pointwise convergence. 



AcB= \J{vhj^O}. 



h=l 

Since sup^ |m| < j, it holds sup^ |w| < sup^ \u\ < j. Thus, each Cj is closed. Let us now 
prove (6.13). Let v G Vu- Since v G L°°(r2), there exists jo > 1 such that \v\ < jo, a.e. in 
Q. Since m is uniformly bounded over {v ^ 0}, up to enlarging Jq, we may as well assume 
that |m| < jo over {v ^ 0}. Hence u G Cj^^, proving the assertion, the converse inclusion 
being trivial. Finally, let us prove that /'(m)|cj is continuous. To this aim, let [vh] C Cj 
with t>/i — !■ f in VFo'^(f2), as /i — ?> oo. For all /i > 1, since Vh G Vu-, by Proposition 6.8, we 
get 

f{u)vh= / jt(n, |Dn|)— -— ■L)i;/,+ / js{u,\Du\)vh+ / |n|P"\t;/, - / |n|''~V;/,. 



Of course, the last two integrals converge to J^ \u\^~^uv and J^ \u\''~'^uv, respectively. 
Also 

jt{u, \Du\)-—— ■ Dvh -)■ jt{u, \Du\)-—— ■ Dv, a.e. in Q, 

\Du\ liJnl 

js{u, \Du\)vh — !■ js{u, \Du\)v, a.e. in Q. 
In addition, in light of the growth conditions on jt,js, for some positive constant Mj, 

Du 

\jt{u, \Du\)-— ■ Dvh\ < Mj\Du\P-'\Dvh\, a.e. in Q, 
\Uu\ 

\3s{u, \Du\)vh\ < MjlDul^lvhl < jMj\Du\^\ a.e. in fi. 

Then, Lebesgue dominated convergence theorem yields f'{u)vh — )■ f'{u)v, as /i — ?■ oo. D 

Then we have the following 

Lemma 6.10. Assume conditions (6.4)-(6.6) and letu G Fix(G7v) such that J{u) < +oo. 
Then, the following facts hold: 

(i) for every v & Vu, we have 

J°{u] v) < J'{u)v. 
(ii) for every t> G Ki H Fix(GAr), we have 
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Proof. The proofs of assertions (i) and (ii) are similar. Hence, let us focus on the proof 
of (ii). Let ?7 > with J{u) < rj. Moreover, let f G Ki ft Fix(G7v) and s > 0. Take now 
r eR with 

(6.14) J'{u)v = f jt{u, \Du\)^ ■Dv+ [ j,(n, \Du\)v < r. 

Jn l^^ Jn 

Let H G C°°(M) be a cut-off function such that H{s) = 1 on [-1, 1], H{s) = outside 
[—2,2] and \H'{s)\ < 2 on M. Notice that, as t" G K, we also have H{^)v G K and 
H{f)v G Vz for all z G Wq'^{Q) and k > 1. Then, by exploiting the growth conditions 
on jt and js, it is possible to prove that there exist k > 1 and 6 > (depending upon k) 
such that 



(6.15) H(^)v-v 

k 



<e, 



as well as 

(6.16) J'{z + i^H{j)v) (h{j)v) < r, 

for all z G B{u,8) fl J'' and d G [0,5). Since the map {t i-)- J{z + tH{j:)v)} is of class 
C^, by applying Lagrange theorem on [0,t] and taking into account (6.16), there exists 
?9 G [0, t] with 

(6.17) J{z + tH{^)v) - J{z) = tJ\z + ^Hfyv) (Hij)v) < rt, 

for every z G B{u,6) H J^ and all t G [0,6). Up to reducing the value of 5 > 0, we may 
also assume that J{u) + 6 < r]. Notice that H{^)v G Fix(GAr) for all v, z & Fix(GAr), since 

ygeGM-. g{Hi^)v)=H{^)gv = H{^)v. 

Then, on account of inequality (6.15), we are allowed to define the continuous function 

n :55(m, J(M))nepi(J)n(Fix(Gjv) xR)x]0,5] ^ ^^(t;) n Fix(Gjv), 
by setting 

n{{z,fi),t):=H{^)v. 

Notice that, for all {z,fj,) G Bs{u, J{u)) fl epi(J), we have z G B{u,6) fl J^. Hence, by 
inequality (6.17), we have 

J{z + tn{{z, /i), t)) < J{z) +rt< fi + rt. 

whenever {z,fi) G Bs{u, J{u)) fl epi( J) and t g]0,5]. Then, according to Definition 2.4, 
we can conclude that ( J|Fix(Gjv))°(w; v) < r. By the arbitrariness of r, it follows that 

{J\Fi^(GN))°e{u;v) < / jt{u,\Du\)——-Dv+ / jsiu,\Du\)v. 
Jn l-f^'"! Jn 

By the arbitrariness of e, we get (J^|Fix(Giv))°(^5 ^) ^ J'{u)v, for all v E Vu- □ 
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Remark 6.11. Under the assumptions of Lemma 6.10, assuming that dJ{u) ^ 0, then 
dJ{u) = {a} for some a G W~^'^ (Q) such that 

\fv G K : / jt{u, \Du\)-—— ■ Dv+ js{u, \Du\)v = {a,v). 

On account of the definition of df{u), it is sufficient to combine (i) of Lemma 6.10, the 
hnearity of {v i— )■ J'{u)v} and, of course, the density of Vu in Wq '^(i^). In a similar fashion, 
assuming 9J|Fix(Giv)('") ¥" 0) then 9J|Fix(Giv)('") — {/^} ^^^ some f3 G (Fix(GAr))' such that 

Vt; G K n Fix(G^) : / j,(m, \Du\)^ ■ Dv + [ j,{u, \Du\)v = {f3, v). 

Jn \Du\ J^ 

This follows by (ii) of Lemma 6.10, the linearity of {v i— )■ J'{u)v} and, finally, by the 
density of \4 fl Fix(G7v) in Fix(G'Ar), which is stated in Proposition 6.15. 

Finally, returning to the functional / defined in (6.2), we have the following 

Corollary 6.12. Assume conditions (6.4)-(6.6) and let u G Fix(G'Ar) such that f(u) < 
+00. Then, the following facts hold: 

(i) for every v ^ V^, we have 

f\u-v)<f{u)v. 

(n) for every t> G \4 H Fix(G'Ar), we have 

(/|Fix(Giv))°(w;^) < f'{u)v. 

Proof. By (6.2), it is / = J + / with / : WI'^{VL) ^ M of class C^. Then /°(n; v) = r{u)v 
for any v E Vu and (J|Fix{Gjv))°(^; ^0 = I'{u)v for any t; G \4 fl Fix(G'Ar). Of course, we 
have J{u) < +oo. Then, by combining [3, Theorem 5.1] with Lemma 6.10, we have 

Vi^GK: f°{u-v) = {J + ir{u-v) 

<J°{u-v) + r{u;v) 

< J'{u)v + I'{u)v = f'{u)v. 

In a similar fashion, we have 

Vw G W G K n Fix(GAr) : (/|Fix(Giv))°(^; V) = (J|Fix(Giv) + ^|Fix(Giv))°(«; v) 

< (J|Fix(G^))°(M;t^) + (/|Fix(G^))°(M;^^) 

< J'{u)v + I'{u)v = f'{u)v, 
concluding the proof. D 

In light of the previous facts, we have the following 

Corollary 6.13. Let f : Wq'^{Q) — t- M U {+C)o} be the G^ -invariant functional defined 
in (6.2) which satisfy assumptions indicated in Section 6.0.2. Let u G Fix(G7v) with 
f{u) E^ he a critical point o//|Fix{Gjv); ^^^^ «s G 9/|Fix(Gjv)(^)- Then 

\/v G K : f'{u)v = 0. 
Furthermore, df{u) = {0} provided that df{u) is nonempty. 
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Proof. It is sufficient to apply Theorem 4.1, since conditions (4.1)-(4.4) are satisfied in 
view of Proposition 6.8, Lemma 6.9 and Corollary 6.12. D 

Next we state some technical facts, necessary to the proof of the main result. Theo- 
rem 6.4. 

Proposition 6.14. It holds 

V(n,0 e epi (/iFix(G^)) : f{u) < ^ ^ M^/lFix(G^)l(^'0 = 1- 

Proof. Notice ffist that [20, Theorem 3.11] extends to the case where Hq{Q) is substituted 
by Wq'^{Q), Q is possibly unbounded and the growths (6.4)-(6.6) are assumed. Since 
f = J + I with I G C^(Fix(G7v),I^), the assertion follows by [20, Proposition 3.7] with 
X = Fix(G7v) by using [20, Theorem 3.11] with J replaced by the restriction J|Fix(Gjv)- 
In fact, it is sufficient to notice that for the deformation (for 6,7] > and k > 1) 

J^{z, t) = (1 - t)z + tTk{z), z e B{u, 5) n J^ t G [0, (5], 

making the job in [20, Theorem 3.11] it is ^{z, t) G Fix(G'w) for z G B{u, 5) n Fix(G^). 

D 

Proposition 6.15. Let u G Fix(GAr). Then the subspace Vu n Fix{G]y) is dense in 
FixiGr,). 

Proof. Let v G Fix(G'Ar) and take e > 0. Let T^ : M — t- M be the Lipschitz function such 
that Tfc(s) = s, for |s| < k, and Tk{s) = ks\s\~^, for |s| > k. Then Tk{v) G Wq'^^Q) fl 
L°°{Q) and Tk{v) G Fix(G7v)- By Lebesgue's theorem, there exists k > 1 such that 
||w — Tfc(t>)||ip < e/2. For this k >1, for all h> 1, consider now Vh = H{u/h)Tk{v), where 
H is defined as in the proof of Lemma 6.10. Then, vy, G WI'^{VL) n L°°{VL), vh G Fix(G'7v) 
and, of course, u G L'^{{vh 7^ 0}). In turn, Vh E VuP\ Fix(GAr), Vh{x) — )■ Tfc(w)(x), 
|'y/i(a^)| < l^fcl"^)! ^ l'^(3^)l for ^-^^ X eVL, DjVh{x) — !> DjTk{v){x) a.e. in fi as /i — > 00 and 
there exists a positive constant C (depending upon /c), such that 

\DjVh{x)\ = \h^^H'{u/h)Dju{x)Tk{v){x) + H{u/h)Djv{x)x{\v\<k}{x)\ 
< C\Dju{x)\ + C\Djv{x)\, for a.e. x eVl. 

By Lebesgue's theorem, we find /i > 1 so large that ||T/c(t;) — Vh\\i,p < e/2. Then, we 
conclude that ||% — f||i,p < |K'?i — 7fc(^)||i,p+||7fc(^) — ^'||i,p < e with % G \4nFix(G'jv). D 

Proposition 6.16. Let u G Fix(GAr). Then it holds 

M/|Fix(Giv)l(M) > sup { / itiu, \Du\)—— -011+ j js{u, \Du\)v 

+ / \u\P~\v- / |n|«"V; : veVuHFixiGN), |K'||i,p < ij- 
Jq Jq ^ 

Moreover, 

(6.18) / jt{u,\Du\)\Du\+ / js{u,\Du\)u< \dJ\Fi^(GN)\{u)\\u\\i,P- 

Jn Jn 

In turn, «/ |<iJ|Fix(G]v)l('") < '^^' then jtiu, \Du\)\Du\ G L^iVt) and js{u, \Du\)u G L^{Q). 
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Proof. Concerning the first assertion, notice that [20, Propositions 4.5 and 6.2] extend 
to the case where Hq{Q) is substituted by Wq'^{Q), Q is possibly unbounded and (6.4)- 
(6.6) are assumed. The assertion follows as in [20, Propositions 4.5 and 6.2] applied 
with / and J substituted by /|Fix(G]v) ^^^ J\Fix{GN)- ^^ i^ enough to notice that, fixed 
V & Vur\ Fix(GAr), the deformation (for 6 > and k > 1) 

Jlf{z,t) = z + j^^H{^)v, z e B{u,6), t e [0,6], 

that makes the job in the proof of [20, Proposition 4.5] satisfies J^{z,t) G Fix(G7v) for 
every z G B{u,5) fl Fix(GAr). Concerning the second assertion, notice that [20, Lemma 
4.6] extends to the case where HliVt) is substituted by WQ'^{Vt), Vt is possibly unbounded 
and (6.4)-(6.6) hold. Furthermore, the deformation (for 5 > and A; > 1) 

J^{z,t) = z- / Tk{z), zeB{u,5), te[0,5], 

||Tfc(n)||i,p(l + e) 

that works inside the proof of [20, Lemma 4.6] is again such that ,^{z,t) G Fix(G'7v) 
whenever one takes z G -B(m, 5) fl Fix(Gjv). D 

Proposition 6.17. Let w G (Fix(GAf))' and let u G Fix(G'Ar) he such that 

is{u,\Du\)v + / it{u,\Du\)-—— ■ Dv + / \u\^~'^uv = I \u\'*~'^uv + {w,v), 
'n Jn l^^ Jq Jn 

for every f G Ki fl Fix(GAr). Moreover, suppose that jt{u, \Du\)\Du\ G L}{VL), and that 
there exist v G Fix(GAr) and rj G L}{VL) such that 

Du 

(6.19) js{u,\Du\)v>ri, jt{u,\Du\)-——-Dv>r]. 

\L>u\ 

Then js{u, \Du\)v G L^(r2), jt{u, |-Dn|)|^^ ■ Dv G L^(r2) and v is an admissible test. 

Proof. It is a simple adaptation of [20, Theorem 4.8] to the invariant framework. D 

We are now ready to provide the proof of Theorem 6.4. 

6.0.5. Proof of Theorem 6.4- By Proposition 6.8 and Corollary 6.12, / satisfies condi- 
tion (4.1) and conditions (4.2) and (4.3) of Theorem 4.1. Furthermore, by Lemma 6.9, 
condition (4.4) is satisfied as well. We now consider / restricted to Fix(GAr), namely 

/|Fix(G^.):Fix(G^)^MU{+oo}. 

Let us now prove that /|Fix(G]v) admits a nontrivial critical point u G Fix(G'7v), by applying 
the Mountain Pass theorem for semi-continuous functionals (cf. e.g. [20, Theorem 3.9]). 
This result requires condition (2.2) to be satisfied, which holds true by Proposition 6.14. 
In light of j{s,t) > aot^, there exist po > sufficiently small and ctq > such that 
f{u) > (To for any u G FixlG^) with ||ti||i,p = Po- Moreover, fixed a nonzero function 
ip G Fix(G7v) n L°°{Q), by virtue of assumption (6.9), we can find a positive number r 
such that 



irWijUr^ < i2qr'\mi\\Di;\\;^T^ and r>max||^|^ 
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In turn, setting Vi := rip G Fix(G7v)5 we obtain ||t'i||i,p > Po and 



/(^'i) < a(r||V'||oo)r^||^^||p + r^ r"? < r^ r"?— — < 0. 

'^ p q p Iq 

Therefore inf{/(u) : u G Fix(G'Ar), ||ti||i,p = Po} > ctq > = max{/(0), /(t'l)}, so that / 
admits a positive Mountain Pass value. Furthermore, for any c &"^, f satisfies the Palais- 
Smale condition at level c. Although the proof is essentially contained in [20], for the 
sake of self-containedness, we shall provide a sketch in the following highlighting the main 
differences. Let then {uh) C Fix(GAr) with f{uh) -^ c and |<i/|Fix(G]v)l(^/i) — >■ 0, as /i — )■ oo. 
Then, by combining Proposition 6.15, Proposition 6.16 and the Hahn-Banach theorem, 
there exists {wh) C (Fix(GAr))' such that Hui/jH-iy — ?■ as /i — )■ oo, jt{uh, \Duh\)\Dufi\ G 
L^{n) (by (6.18), since M/|Fix(Giv)l(^*/^) < +^ yields |rfJ|Fix(GAr)l(M) < +oo in light of [3, 
Theorem 4.13(ii) combined with Theorem 5.1(ii)]) and 

(6.20) / js{uh, \Duh\)v + / jt{uh, \Duh\) ^ ■ Dv 

IP-2,,.,, _ f u,. i<?-2 



+ / \Uh\^' UhV = I \Uh\'^ UhV+{Wh,v), 

Jn Jn 

for every v G Vu^ fl Fix(G'jv). By assumption (6.8), the sequence (uh) C Fix(G'7v) is 
bounded in Fix(G7v) (see e.g. [22, Lemma 3.15]). Moreover, by means of Proposition 6.17, 
due to (6.7) we can choose the test v = Un in (6.20), yielding 



js{uh,\Duh\)uh+ / jt{uh,\Duh\)\Duh\+ / \uh\^ = / luhl" + {wh,Uh). 
i Jn Jfi Jn 

By the sign condition (6.7) and the boundedness of (uh) in Wq'^{Q), this yields 

(6.21) sup / jt{uh, \Duh\)\Duh\ < +00, sup / js{uh, \Duh\)uh < +oo. 

h>i Jn h>i Jn 

We claim that, up to a subsequence, (uh) is strongly convergent to some u G Fix(G'Ar). In 
this step, of course, the compact injections (6.1) will play a crucial role. Let u G Fix(G7v) 
be the weak limit of the sequence (uh) in Fix(G7v)- Since jt{s, t)t > 0, by Fatou's Lemma 
the first formula in (6.21) yields jt{u, \Du\)\Du\ G L^(f2). 

By following the first part of the proof of [20, Theorem 5.1], we get Duh{x) — t- Du{x) 
a.e. in Q. More precisely, given a sequence of bounded GTv-invariant invading domains 
{^k) of ^) one is lead to the application of [7, Theorem 5] to a suitable variational identity 
involving a class of Leray-Lions operators bh{x,^) related to j(s, |^|), (wh) C (Fix(GAr))' 
convergent in || ■ ||-i,p' and a sequence (fh) bounded in L^(f2fc). This is achieved by testing 
(6.20) with H{uh/k)v G Vu,^ nFix(G'7v)! where v G WQ'''\VLk) is any bounded GTv-invariant 
function. In fact, after suitably rearranging the terms arising in the calculation, one is 
lead to 

/ hh{x, Duh) ■ Dv = {wh, v) + / fhV, 
J^k J^k 

for every Gjv-invariant function v G WQ'^{Qk) H L°°{Qk)- 

We now point out that, in our framework, differently from [7] (cf. [7, Formula 19]) the 
above variational identity only holds for Gjv-invariant test functions. On the other hand. 
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since Qk and Uh are GTv-invariant, this is actually sufficient to succeed. In fact, for a 
suitable sequence of numbers 6k > 0, the only test functions which arise in the proof of 
[7, Theorem 5] are indeed GTv-invariant functions, being exactly of the form 

Vh := ^K i^s, o {uh -u)e W^o'^lOfc) n L°°(fifc), 

where K is a. fixed GTv-invariant compact subset of Qk, fK ^ C^{Qk) with < (px < 1 
is a GAT-invariant function with ipx = 1 on K and supt((/9i^) C f/ C f/ C ^2^ for some 
(jTV-invariant open set U (^ Qk and ips G C^(]R), ips{s) = s for {|s| < 6}, ips{s) = for 
{\s\ > 26}, \ips\ < 26 and \Dips\ is bounded. Due to the GTv-invariance of ^2^ restricting to 
(jTV-invariant compacts K of fik is without loss of generality. In conclusion, [7, Theorem 5] 
allows to conclude that Duh{x) — )■ Du{x) for a.e. x in the set Ek = {x & Vtk '■ \u{x)\ < k}. 
Finally, the property on Vt follows by a standard diagonal argument on the EkS. Moreover, 
taking suitable test functions in Fix(G7v) (cf. [20, test (5.8) and the test above (5.11)]), 
we get 

(6.22) / j,(n, \Du\)v + / ]t{,u, \Du\)—^ ■ Dv + f \u\p~\v = f |n|^~\w, 

for every f G Kt H Fix(G'Ar). For M > 0, consider the Lipschitz function (^ : M — t- R"*", 

(6.23) C(s) = M\s\ for |s| < R, C(s) = MR for |s| > R. 

By (6.7), the growth condition on jg and jj(s, t)t > aot^, we can easily choose M (depend- 
ing upon R and ao) such that [js{s,t) + jt{s,t)t('{s)]s > for all s G M and t G M"*". Re- 
calling that jt{uh, \Duh\)\Duh\ G L^{Q) and jt{u, \Du\)\Du\ G L^{^), by Proposition 6.17, 
condition (6.7) and the definition of (, the function Vh '■= m/iC''^"''^ G Fix(G7v) (resp. 
V := ue''^"-' G Fix(GAr)) can be taken as admissible test functions into (6.20) (resp. in 
(6.22)). Finally, since by virtue of Lemma 6.2 (uh) strongly converges to u in L'^{Q), by 
dominated convergence. 



lim / |«ft|«e^("'^) = / |m|V("). 
^ Jn Jn 

Therefore, using Fatou's Lemma twice, we reach 

ji(u, |Dn|)|Dn|e^(")+ /" |n|Pe^(") 

Jn 

j,(n,, |Dn,|)|Dn,|e«"'^) + / |n,|V("'^)" 

Jn -^ 

jt{uh, |Z^n,|)|Z}n,|e'^("'^) + / In^^e^"'^) 
! Jn ^ 

'^ Jn 



< liminf 

h 

< lim sup 

h 



n 



f 

Jn 



< / |M|«e«")- / [js{u,Du)+jt{u,\Du\)\Du\C{u)]ue<'-^^ 
Jn Jn 

\Du\)\Du\e^^''^ + [ |M|Pe'^(") 



n 
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which combined with the pointwise convergence of the gradients and the inequahties 

imphes that ||m/i||p — j- H'^^Hp and HZ^-U/dlp -^ \\Du\\p as /i — > oo, by dominated conver- 
gence. By the uniform convexity of L^, we conclude Uh -^ u in Wq'^{Q) as /?, — )■ oo. 
The Palais-Smale condition is thus verified. By applying [20, Theorem 3.9], we find a 
nontrivial Mountain Pass critical point u e Fix(G7v), namely |(i/|Fix(Giv)l('") — 0- ^^ turn, 
in light of Proposition 2.6, G <9/|Fix(Giv)('^)- Hence, by Theorem 4.1 (more precisely by 
Corollary 6.13), we get 

yv G K : f'{u)v = 0. 
Now, by Proposition 6.8, we have 

In light of Proposition 6.16, it follows jt{u, \Du\)\Du\ G L^{Q) and js{u, \Du\)u G L^{Q). 
Then, it holds \js{u, \Du\)\ < \js{u, \Du\)\x{\u\<i} + \js{u, \Du\)u\. Also, for a fixed com- 
pact K G Q, it holds 

\jt{u, \Du\)\xk < \jt{u, \Du\)x{\du\>i}Xk\ + \jtiu, \Du\)x{\du\<i}Xk\ 

< jtiu,\Du\)\Du\ + Ca{\u\)xK 

< jt{u, \Du\)\Du\ + C\uf + CxK- 

We have thus proved that jsiu, \Du\) G L^{VL) and jt(M, \Du\) G L\^^{yt). In turn, for any 
V G C^°°(^), we have i,(M, \Du\)v G L^{Vt) and ji(M, |/^m|)^ ■ Dv G ^^(1]), so that, in 
light of [20, Theorem 4.8] (which extends to the current setting), we get 

Vt; G Cr(^) : / 3t{u, \Du\)^^ ■Dv+ f j,{u, \Du\)v + f \u\p~\v = f \u\''~\v, 
Jn l"^l Jq Jn Jn 

namely m is a distributional solution. The proof is now complete. D 

6.0.6. Proof of Corollary 6.5. Let Vt = R^ and take X = W^'^iE.^) and G = ff{N). Then, 
the assertion follows directly from Theorem 6.4 since the invariance u{gx) = u{x) for all 
(7 G G is equivalent to the radial symmetry of u. To obtain that u is nonnegative, in the 
application of Theorem 6.4 it suffices to replace, in the definition of /, the term J^i^ 1^1^/? 
with J^j^G{u), where G{s) = {s^Y/q. Hence, as in the proof of Theorem 6.4, after 
application of our second Palais symmetric criticality principle, one finds a m G Fix(G) 
which solves 

jt{u,\Du\)-—— ■ Dv + I is{u,\Du\)v + I \u\^'''^uv = I \u^\''^'^uv, 

\DU\ J^N J^N J^N 

for all V G Vu- If C is the map defined in (6.23), the function v = — n^e^^"-* can be taken 
as an admissible test function in the above identity by [20, Theorem 4.8] in view of the 
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growth assumptions on js,jt and of the sign condition (6.7), yielding 
0= /" \u+\''~'^uv= f e^^'''^jt{u,\Du^\)\Du~\ 
+ [ -[jsiu,\Du\)+jt{u,\Du\)\Du\Ciu)]u-e'^^''^ + f |m-|V('') 
> f |m-|V(") > f |m^|^>0, 

yielding m~ = 0, hence m > 0. This concludes the proof. D 

Acknowledgment. The author thanks Marco Degiovanni for some very useful discus- 
sions. 



[1 
[2 
[3 

[4; 

[6 

[7; 

[8 

[9 

[10 

[11 

[12 

[13 

[14 
[15 



References 

T. Bartsch, M. Willem, Infinitely many nonradial solutions of a Euclidean scalar field equation, 
J. Fund. Anal. 117 (1993), 447-460. 

H. Brezis, Symmetry in nonlinear PDE's. Differential equations: La Pietra 1996 (Florence), 1-12, 
Proc. Sympos. Pure Math., 65, Amer. Math. Soc, Providence, RI, 1999. 

I. Campa, M. Degiovanni, Subdifferential calculus and non-smooth critical point theory, SIAM 
J. Optim. 10 (2000), 1020-1048. 

F.H. Clarke, Optimization and non-smooth analysis, Canadian Mathematical Society Series of 
Monograph and Advanced Texts, John Wiley & Sons, New York, 1983. 

S. Coleman, Classical lumps and their quantum descendants, in: Zichichi A., Editor, New Phenom- 
ena in Subnuclear Physics. Proceedings of the first half of the 1975 International School of Subnuclear 
Physics, Erice, Sicily, July 11 - August 1, 1965, vol. A, 297-407, New York, Plenum Press. 
J.N. CORVELLEC, M. Degiovanni, M. Marzocchi, Deformation properties for continuous func- 
tionals and critical point theory, Topol. Methods. Nonlinear Anal. 1 (1993), 151-171. 
G. Dal Maso, F. Murat, Almost everywhere convergence of gradients of solutions to nonlinear 
elliptic systems. Nonlinear Anal. 31 (1998), 405-412. 

M. Degiovanni, M. Marzocchi, A critical point theory for non-smooth functionals, Ann. Mat. 
Pura Appl. 167 (1994), 73-100. 

M. Degiovanni, S. Zani, Euler equations involving nonlinearities without growth conditions, Po- 
tential Anal. 5 (1996), 505-512. 

J. KOBAYASHI, M. Otani, The principle of symmetric eritieality for non-differentiable mappings, 
J. Funct. Anal. 214 (2004), 428-449. 

O. Ladyzenskaya, On finding symmetrical solutions of field theory variational problems. Proceed- 
ings of the International Congress of Mathematicians, Vol. 1, 2 (Warsaw, 1983), 1315-1329, PWN, 
Warsaw, 1984. 

W. Krawcewicz, W. Marzantowicz, Some remarks on the Lusternik-Schnirelman method for 
nondifferentiable functionals invariant with respect to a finite group action. Rocky Mountain J. Math. 
20 (1990), 1041-1049. 

A. Kristaly, C. Varga, V. Varga, A nonsmooth principle of symmetric eritieality and 
variational- hemivariational inequalities, J. Math. Anal. Appl. 325 (2007), 975-986. 
P.L. Lions, Symetrie et compacite dans les espaces de Sobolev, J. Funct. Anal. 49 (1982), 315-334. 
P.-L. Lions, The concentration-compactness principle in the calculus of variations. The locally 
compact case. I. Ann. Inst. H. Poincare Anal. Non Lineaire 1 (1984), 109-145. 



26 MARCO SQUASSINA 

[16] P.-L. Lions, The concentration-conipactncss principle in the calculus of variations. The locally 
compact case. II. Ann. Inst. H. Poincare Anal. Non Lineaire 1 (1984), 223-283. 

[17] R.S. Palais, The principle of symmetric criticality, Comm. Math. Phys. 69 (1979), 19-30. 

[18] R.S. Palais, Applications of the symmetric criticality principle in mathematical physics and dif- 
ferential geometry. 1981 Shanghai symposium, diff geometry and diff equations, 247-301, Science 
Press, 1984. 

[19] R.S. Palais, Symmetry, criticality, and condition C. The Sophus Lie Memorial Conference (Oslo, 
1992), 77-98, Scand. Univ. Press, Oslo, 1994. 

[20] B. Pellacci, M. Squassina, Unbounded critical points for a class of lower semicontinuous func- 
tionals, J. Differential Equations 201 (2004), 25-62. 

[21] M. Squassina, Radial symmetry of minimax critical points for non-smooth functionals, Commun. 
Contemp. Math., to appear. 

[22] M. Squassina, Weak solutions to general Euler's equations via non-smooth critical point theory, 
Ann. Fac. Sci. Toulouse Math. 9 (2000), 113-131. 

[23] M. Squassina, Existence, multiplicity, perturbation, and concentration results for a class of quasi- 
linear elliptic problems. Electron. J. Differential Equations Monograph 7 2006 +213 pp., Texas State 
University of San Marcos, San Marcos, Texas, TX, USA. 

[24] W.A. Strauss, Existence of solitary waves in higher dimensions. Comm. Math. Phys. 55 (1977), 
149-162. 

[25] J. Van Schaftingen, Symmetrization and minimax principles, Commun. Contemp. Math. 7 
(2005), 463-481. 

[26] H. Weyl, Space-time-matter, New York, Dover, 1951. 

[27] M. WiLLEM, Minimax theorems, Birkhauser Verlag Basel, 1996. 



DiPARTIMENTO DI INFORMATICA 

Universita degli Studi DI Verona 

Ca Vignal 2, Strada Le Grazie 15, 1-37134 Verona, Italy 

E-mail address: marco.squassina@univr.it 



